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ABSTRACT 

Objective To compare Orchard Sports Injury 
Classification System (OSICS-1 0) sports medicine 
diagnoses assigned by a clinical and non-clinical coder. 
Design Assessment of intercoder agreement. 
Setting Community Australian football. 
Participants 1082 standardised injury surveillance 
records. 

Main outcome measurements Direct comparison of 
the four-character hierarchical OSICS-1 0 codes assigned 
by two independent coders (a sports physician and an 
epidemiologist). Adjudication by a third coder 
(biomechanist). 

Results The coders agreed on the first character 95% 
of the time and on the first two characters 86% of the 
time. They assigned the same four-digit OSICS-1 0 code 
for only 46% of the 1082 injuries. The majority of 
disagreements occurred for the third character; 85% 
were because one coder assigned a non-specific 'X' 
code. The sports physician code was deemed correct in 
53% of cases and the epidemiologist in 44%. Reasons 
for disagreement included the physician not using all of 
the collected information and the epidemiologist lacking 
specific anatomical knowledge. 
Conclusions Sports injury research requires accurate 
identification and classification of specific injuries and 
this study found an overall high level of agreement in 
coding according to OSICS-1 0. The fact that the majority 
of the disagreements occurred for the third OSICS 
character highlights the fact that increasing complexity 
and diagnostic specificity in injury coding can result in a 
loss of reliability and demands a high level of anatomical 
knowledge. Injury report form details need to reflect this 
level of complexity and data management teams need to 
include a broad range of expertise. 



INTRODUCTION 

Injury data collection or surveillance is the founda- 
tion of most sports injury prevention research. 1 2 
Injury data collected in a routine, systematic 
manner, when coupled with appropriate exposure 
data, can (1) enable estimation of injury incidence 
in both experimental and observational studies, 
(2) be used to monitor temporal and spatial trends 
in injury rates and (3) be used to evaluate the 
success of injury prevention strategies. 3 The 
amount and type of data that are collected in an 
injury surveillance system is largely dependent on 
the needs of its end-users (eg, clinicians, epidemiol- 
ogists, etc). However, it is also recommended that a 
core set of variables should always be collected, 
regardless of the system's purpose. 4 5 



The quality of data collected for sports medicine 
is dependent on how well key data items, such as 
injury diagnoses, nature and body part are recorded 
and coded. Use of standardised coding systems 
enables cross-study comparisons. Unfortunately, 
many important sports medicine diagnoses are com- 
pletely absent from the International Classification 
of Diseases (ICD). 6 

Several diagnosis coding systems have been devel- 
oped to meet the specific needs of sports injury 
research. The Sport Medicine Diagnostic Coding 
System (SMDCS), first developed in 1991 for use in 
the Canadian Intercollegiate Sport Injury Registry, 
has since been incorporated into the National 
Collegiate Athletics Association Injury Surveillance 
System, the National Hockey League and the 
Canadian Athlete Monitoring Program. 7 To date, 
only one paper has been published on the SMDCS 
itself 7 and the system's reliability and validity are 
unknown. 

The Orchard Sports Injury Classification System 
(OSICS) which is a hierarchical four-character system 
for coding sports medicine diagnoses has been more 
widely applied. 8 Originally developed in 1992 for use 
in elite Australian football, it is now in its 10th 
version (OSICS-1 0, http://www.johnorchard.com/ 
osics.html). It is recommended by international sports 
injury consensus statements, 9-13 and has been incor- 
porated into the injury surveillance systems of many 
international professional sporting bodies. 14-19 It has 
been suggested that the high level of detail required in 
the new four-character version of OSICS-1 0 could 
potentially increase the capacity for disagreements 
between users and that the system is too complex for 
users without a clinical background. 18 

As with any outcome measure, injury diagnosis 
coding schema must have good reliability to ensure 
comparability across studies and clinicians/data 
coders. In many studies, the person who diagnoses 
or records the injury is not the same as the person 
who codes the injury for database management 
purposes. This introduces potential error because a 
coder could either misinterpret the written diagno- 
sis or select the wrong code. 6 The likelihood of this 
can be increased for coders with different back- 
grounds or levels of expertise. While there is some 
evidence that having a non-clinical background 
may affect the accuracy of coding sports injuries 
using the ICD, 6 this has not previously been evalu- 
ated for OSICS. Therefore, this study aimed to 
evaluate intercoder agreement in OSICS coding 
between individuals with and without a clinical 
background. 
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METHODS 

The OSICS-10 is structured hierarchically with four characters 
assigned to each injury. 20 The first (leftmost) character relates to 
the anatomical location; the second character to the specific 
injured tissue or the pathology of the injury; the third and 
fourth characters further describe the pathology or broaden the 
diagnosis. 19 20 The letter X is used as a character when some 
aspect of the injury is unspecified, 20 or as the fourth character if 
sufficient detail is provided by the previous three characters (eg, 
the code AFAX is used to denote a fracture of the tibia and 
fibula at the ankle joint, because this is fully specified by the first 
three characters). The letter Z is used to denote an absence of 
injury or illness details or to represent some 'other' injury not 
listed by OSICS (eg, the code AFTZ indicates a fracture to the 
talus of the ankle, not further specified). 19 

The injury data used in this study were taken from the 
Preventing Australian Football Injuries through eXercise (PAFIX) 
randomised controlled trial (RCT). Full details of the study 
protocol are published elsewhere 21 and the injury data collection 
procedures shown to be reliable. 22 The study was approved by 
the University of Ballarat Ethics Committee. In summary, 40 
trained primary data collectors (PDCs) with an exercise and 
sports science background used a standardised data collection 
form (figure 1) to collect details of injuries sustained during all 
games and training sessions during the 2007 and 2008 playing 
seasons in 18 community-level Australian football clubs. 

Overall, 1082 injury record forms were completed by the 
PDCs. No attempt was made to confirm injury details with a 
medical practitioner as many injuries would only have been 
treated by the first-aid staff at the game. 

The full injury records were checked and edited before 
exporting to a Microsoft Excel spreadsheet. This was then 



provided to two independent coders for assigning of an OSICS 
code to each injury record. One coder was a sports physician, 
with extensive experience in the use of OSICS and the diagnosis 
of sports injuries. The second coder was an epidemiologist with 
no clinical background, but with considerable experience in the 
analysis and reporting of sports injury data, including with 
OSICS. Each coder entered the OSICS-10 codes they allocated 
into separate versions of the Excel spreadsheets. 

Both sets of allocated codes were compared to identify exact 
matches and discrepancies. As OSICS-10 is hierarchical, it was 
considered only relevant to compare disagreements at a given 
character level, if coders agreed fully on all preceding characters. 
For example, it was not relevant to determine how many codes 
disagreed at the second character level if they did not agree at 
the first character level. A third coder, with a background in 
injury biomechanics but no previous experience in OSICS 
coding, adjudicated when the two main coders differed. The 
adjudicator, who was blinded to the coder backgrounds, decided 
on the final OSICS code to be assigned. The nature of the sys- 
tematic discrepancies were documented and summarised by the 
adjudicator; they were then interpreted in the light of the coder 
background, but only after the adjudication was finalised. 

All data are presented as the percentage (%) of agreements or 
disagreements. Patterns in the reasons for discrepancies were 
noted by the adjudicator and are presented in the results section. 

RESULTS 

The two coders provided exactly the same four-digit OSICS-10 
code for 46.3% (n=501) of the 1082 recorded injuries. The 
53.7% of injuries (n=581) with coding discrepancies underwent 
adjudication. In just over half (54.4%, n=316), the initial four- 
character code assigned by the sports physician was deemed 



PRFft I Injury Incident Report Form 



Office Use Only 
Today's date: I 7 08 



Game Details 



Player Name^ 

Club 

Grade 

Date 

Venue 



| | Training □ Game - 



In which quarter did the injury occur? 

□ Q1 DQ2 DQ3 DQ4 
Position played at time of injury 

| | Forward Line 

| | Back Line 

□ Midfield 



Where on the field did the injury occur? 


(Mark withX). If during game, indicate 


where you were positioned using an 


asterisk. 






4) 













Did the player sustain more than one injury 
during this injury event? 

e.g. sprained ankle and knee abrasion 

□ Yes □No 
What was the intent of the injury? 

I I Within game rules □ Unsure 
I | Outside game rules □ Other 



Body Region Injured 



Head & Neck 

Face (including mouth) 

Teeth/Mouth 

Shoulder 

Upper Arm " 

Elbow 

Lower Arm 

Hand/Fingers 

Chest 

Trunk/Abdomen 

Groin/Hip 

Back 

Backside/Gluteals 
Upper Leg | 
Knee 

Lower Leg ~ 

Ankle 

Foot/Toes 

Other P 



Nature of Injury 

□ Abrasion/grazing 

□ Cartilage injury 

□ Cork/bruise 
Cut/laceration 
Concussion 
Dental 
Dislocation 
Fracture (not stress) 
Rupture I 



Strain (muscle/tendon) 
Sprain (ligament/joint) 
Unsure 
Other f~ 



Cause of Injury 
Contact 

1st point of body contact | 

□ Being tackled 

□ Collision with other player/umpire 

□ Collision with fixed object 

□ Fall/trip/slip on same level during or after 
contact 

□ Struck by ball 

□ Struck/hit whilst attacking 

□ Struck/hit whilst defending 

□ Tackling another player 
Non-contact 

□ Aggravation of previous injury 

□ Fall/trip/slip on same level 

□ Kicking the ball 

□ Landing from jump/mark 

□ Overuse/gradual onset 

□ Overextended 

□ Rapid change in speed (acceleration/ 
deceleration) 

□ Side stepping/cutting 

□ Twisting or changing direction 

□ Other 



Primary Data Recorder Details 

PDC Name ) 

PDC 

Signature 



Injury Details 



Give a description of what occurred at the time 
of injury. Please include details on how many 
players were involved, form of contact, if any, and 
specific movement of injured player. Continue 
overleaf if needed. 



Other Details 



Did the injured player leave the field as a result 
of the injury? 

□ Yes □No 

Did the injured player return to the field after 
receiving medical assistance at the ground? 

□ Yes □No 

If Yes, specify medical attendee 

I I Doctor I I Sports Trainer I I Physio 

□ Other I I 



Was the injured player advised to seek off-field 
medical advice? 

□ Yes □No 

Was the injured player taken to hospital? 

□ Yes □No 



Figure 1 Injury data collection form used in the Preventing Australian Football Injuries through eXercise study. 
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Table 1 Summary of the hierarchical Orchard Sports Injury 
Classification System OSICS-10 coding agreements (n-1082 injuries) 






Agreeing 


Disagreeing 






cases 




cases 




Character level 


n 


n 


% 


n 


% 


First 


1082 


1032 


95.4 


50 


4.6 


Second (when agreed on first) 


1032 


930 


90.1 


102 


9.9 


Third (when agreed on first and second) 


930 


531 


57.1 


399 


42.9 


Forth (when agreed on first, second 


531 


501 


94.4 


30 


5.6 


and third) 















correct; in 44.2% (n=257) of cases, the initial code assigned by 
the epidemiologist was considered correct. The adjudicator dif- 
fered from both coders on eight cases. 

Table 1 shows the level of coding agreements and disagree- 
ments at each hierarchical level of OSICS-10. There was very 
high agreement at the first character level and 84% of all cases 
agreed to the second (given the first was correct) character level. 
The majority of disagreements occurred for the third character 
(given agreement on the first two characters), with only 49% 
agreement. However, when the first three characters agreed it 
was most likely that the fourth would also agree. 

Anatomical location (first character) disagreements 

Overall, there were 50 cases in which the two coders disagreed 
on the first character. The adjudicator judged the sports physi- 
cian's coding to be correct in 80% of these cases and, in one 
case, the adjudicator disagreed with both coders. Of these 50 
disagreements, 32 were because one coder used an X rather 
than a more specific body code. For example, for 30 discrep- 
ancy cases, the text read 'back'; the epidemiologist assigned a 
general code but the physician coded the body region to the 
lumbar spine based on other information given in the written 
record. In five cases, the epidemiologist coded a writs/hand 
injury but the physician correctly coded them to forearm. 

Injured tissue/pathology (second character) disagreements 

After agreeing on the first character, the coders disagreed on the 
second OSICS character in 102 injuries; 5 instances were due to 
one of the coders assigning an X and 40 cases due to assigning 
of a Z. According to the adjudicator, the epidemiologist was 
incorrect in 38 cases, compared to 66 cases for the sports phys- 
ician; the adjudicator disagreed with both in two cases. 
Generally, when the sports physician was deemed incorrect, it 
appeared that additional information present in the injury 
records had not been used. For example, the major body region 
data field category was upper leg but information from a separ- 
ate data field needed to be consulted to indicate the hamstrings 
or quadriceps muscle. There were 15 injuries for which the epi- 
demiologist assigned a non-specific injury code, but the sports 
physician correctly coded them as a sprain or haematoma. 

Specific diagnosis (third and/or fourth character) differences 

After agreeing on both first and second characters, the coders dis- 
agreed on 399 injury diagnoses at the third character level; the 
vast majority (85%) of these was because one coder assigned an X 
and 6% were due to one code assigning a Z. This was the character 
level where the major level of discrepancy between the two coders 
was reported. The adjudicator confirmed that the sports physician 



correctly coded 56.1% (n=224) of cases, compared with 43.1% 
(n=172) for the epidemiologist; they disagreed with both coders 
for three cases. The major reasons for the discrepancies appeared 
to be non-use of additional information present in the injury 
records by the sports physician and a lack of detailed anatomical 
or clinical knowledge on the part of the epidemiologist. Correct 
coding of haematomas and ankle sprains, in particular, was more 
common for the sports physician. 

The two coders only disagreed on the fourth character in 30 
injuries (after agreeing on the first three characters); in two 
cases, the adjudicator disagreed with both coders. The sports 
physician was deemed correct in 17 cases and the epidemiolo- 
gist in 11 cases. In all cases, the discrepancies occurred because 
an X was used by only one coder; in four cases, one coder also 
used a Z. 

Use of X and Z codes 

The OSICS-10 allows for un-needed characters (eg, when the 
diagnosis is fully described by three characters and so a fourth is 
not needed) to be indicated by an X. In the final consensus diag- 
noses of the 1082 injuries, an X was assigned to just the fourth 
character in 959 (or 88.6%) of cases. In 406 cases, the last two 
characters were assigned as XX and in only three cases were the 
last three characters an XXX. When there is insufficient detail a 
Z code can be used. In the final consensus codes, no first charac- 
ter had a Z, 21 second characters were a Z, 15 cases had a Z in 
the third character and only three cases had a Z in the fourth. 

DISCUSSION 

Sports injury research requires accurate injury classification to 
ensure that it is useful for identifying patterns in injury occurrences 
and opportunities for prevention. The accuracy of this coding 
depends on both the quality of the initial information provided 
(eg, text narrative descriptions of injury events can be useful for 
providing information about sports injuries, but can vary in 
quality 23 ) and the translation of this information into an appropri- 
ate code. In many epidemiological field-based data collections in 
non-professional sport, responsibility for data collection, coding 
and/or analysis often rests with a database manager/epidemiologist 
without formal sports medicine clinical knowledge. 

To our knowledge, this is the first study to compare how dif- 
ferent coders allocate OSICS-10 sports medicine codes to a 
large-scale epidemiological database of injuries reported in com- 
munity sport. The injury data collectors were formally trained 
in the data collection procedures and previous research using 
the same sorts of data collectors has been shown to be reli- 
able. 22 24 The information they recorded was then provided to 
two 'expert' coders from different backgrounds and adjudicated 
by a third. 

There was very high agreement at the first character level 
(95%) across 1082 injuries, and 86% of injury codes also 
agreed to the two character OSICS-10 level. While the 
OSICS-10 has four characters to enable a high level of diagnos- 
tic specificity, this level of detail is generally not needed in non- 
clinical settings. 25 The majority of coding disagreements 
occurred for the third OSICS character (given agreement on the 
first two) with only 49% of all cases agreeing at the third char- 
acter hierarchical level. This highlights the fact that increasing 
complexity and diagnostic specificity in injury coding can result 
in a loss of accuracy. 18 25 26 In epidemiological studies it is rare 
to code injuries to the highest level of diagnostic specificity. 
Importantly, given the hierarchical nature of OSICS-10, even 
where the third or fourth characters disagree, the final injury 
code will map to the same two-character category (eg, ankle 
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sprain), which is more commonly reported. It should be pointed 
out, however, that the data collection form was designed to 
maximise information for the first and second character levels 
because the data collectors were often non-clinical people asso- 
ciated with community football teams. There were no specific 
prompts to record additional information that may have assisted 
in assigning more correct codes, such as the type of treatment 
provided. It could be expected that these data recording limita- 
tions could be overcome in studies in professional sport. 

The level of three character coding agreement was lower in 
this study than in previous studies which have evaluated interco- 
der agreement among clinicians. One study had at least two of 
three clinically trained coders assigning three-character OSICS-8 
codes to 300 sports medicine diagnoses and found a coding 
agreement of 84%. 6 In another study, 18 20 written injury diag- 
noses from professional football, cricket and rugby were used to 
assess intercoder agreement among eight clinicians (orthopaedic 
surgeons, physiotherapists and specialist general practitioners). 
Similar to our findings, the level of agreement declined with 
increasing diagnostic specificity with kappa values ranging from 
0.95 (almost perfect agreement) for agreement on the first char- 
acter to 0.56 (moderate agreement) on the fourth character. 
Reasons for the lower agreement in this study could include the 
smaller number of raters and their disparate backgrounds. 

Both coders were provided with exactly the same information 
to code from in the form of an Excel spreadsheet which con- 
tained all information reported on the initial paper-based collec- 
tion form. However, the adjudication of coding disagreements 
by a biomechanist identified some apparent systematic biases in 
how the two coders assigned OSICS codes. First, the physician 
did not always use the additional information in other parts of 
the injury record. This emphasises the importance of ensuring 
that coders consider the full text fields when assigning codes, 
since the text may contain valuable information to enhance the 
injury classification. This issue has also been noted as being 
important for coding injury-related hospital admissions. 27 
Second, the epidemiologist appeared to assign incorrect codes 
based on a lack of detailed anatomical or medical knowledge. 
The fact that different professional backgrounds of coders influ- 
ence the quality of diagnostic coding allocation has also been 
shown in hospital settings. 28 29 

One advantage of OSICS- 10 over previous versions is that its 
hierarchical arrangement of characters allows mapping back to 
broader categories of anatomical location and body structure. 
Even when there is poor coder agreement on the third and 
fourth characters, classifying injuries according to their first and 
second characters can still minimise errors. 18 This study demon- 
strated very high agreement on both the first and second charac- 
ters. It is recommended that when coders have non-clinical 
backgrounds that only the first two characters are used to ensure 
the injury is correctly identified. In most epidemiological appli- 
cations, this two-character level of coding would be sufficient 
for injury description and trend monitoring. 

Coders with non-clinical backgrounds generally have a lack of 
specific anatomic or diagnostic knowledge and may benefit from 
receiving formal training in some aspects of this to improve 
their coding accuracy. In the Australian Football League (AFL) 
Injury Surveillance System coordinated by author JO, sports 
medicine staff at each professional AFL teams prospectively 
record player injury status and the full medical diagnosis of each 
injury. An OSICS code is then assigned centrally and sent back 
to the club for confirmation and corrected, if necessary. While 
this approach may be suitable for professional sport with 



medical support teams, it is unlikely to be appropriate for all 
settings, including community sport. 

The major limitation of this study is that only two coders 
were involved, with a third adjudicating when discrepancies 
occurred. However, the non-clinical coder backgrounds are 
typical of the researchers who would most commonly code 
injuries in large-scale epidemiological studies, particularly with 
the frequent absence of medical or allied health professionals in 
non-professional sport. There could be scope to develop a list 
of injury categories that could be used by non-clinicians to aid 
better coding across several sports. The strength of the study 
was the large number of injuries coded and these covered a 
spectrum of severities from epistaxis to suspected fractures. As 
all data were collected on the field where the injuries occur, it is 
likely that many overuse injuries were not captured and so this 
evaluation is probably most relevant for the coding of acute or 
traumatic injuries. 

CONCLUSIONS AND RECOMMENDATIONS 

In summary, this study has found an overall high level of agree- 
ment in hierarchical OSICS- 10 coding to the two character 
level, which would be appropriate for most epidemiological 
applications. In studies requiring more diagnostic specificity (ie, 
to the third or fourth character level), one beneficial strategy 
could be to ensure that a panel of experts reviews identified 
cases to obtain consensus on their coding. Irrespective of the 
post-data collection coding, it is strongly recommended that 
future injury data collections ensure the recording of high- 
quality text-fields for recording additional detail, over and 
above what might be requested with prespecified tick box 
options. Such information could provide additional information 
about the nature of injury, or the actual treatment received, to 
improve specific diagnoses. This is consistent with narrative text 
descriptions being recommended as core-items of any injury sur- 
veillance system. 4 5 



What this study adds 



► This study emphasises the importance of the background 
expertise of data coders in ensuring sports medicine 
diagnoses are fully accurate and with a high degree of 
diagnostic specificity. 

► Where possible, teams of coders with varying expertise 
should be used to agree on sports medicine diagnoses in 
epidemiological studies. 

► Data coders with no clinical background or limited 
anatomical knowledge can accurately use the OSICS codes 
to the two digit level, which would be appropriate for most 
epidemiological applications. 
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